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Welcome  to  our  Ninth  Annual  Acquisition  Research  Symposium!  This  event  is  the 
highlight  of  the  year  for  the  Acquisition  Research  Program  (ARP)  here  at  the  Naval 
Postgraduate  School  (NPS)  because  it  showcases  the  findings  of  recently  completed 
research  projects — and  that  research  activity  has  been  prolific!  Since  the  ARP’s  founding  in 
2003,  over  800  original  research  reports  have  been  added  to  the  acquisition  body  of 
knowledge.  We  continue  to  add  to  that  library,  located  online  at 

www.acquisitionresearch.net,  at  a  rate  of  roughly  140  reports  per  year.  This  activity  has 
engaged  researchers  at  over  60  universities  and  other  institutions,  greatly  enhancing  the 
diversity  of  thought  brought  to  bear  on  the  business  activities  of  the  DoD. 

We  generate  this  level  of  activity  in  three  ways.  First,  we  solicit  research  topics  from 
academia  and  other  institutions  through  an  annual  Broad  Agency  Announcement, 
sponsored  by  the  USD(AT&L).  Second,  we  issue  an  annual  internal  call  for  proposals  to 
seek  NPS  faculty  research  supporting  the  interests  of  our  program  sponsors.  Finally,  we 
serve  as  a  “broker”  to  market  specific  research  topics  identified  by  our  sponsors  to  NPS 
graduate  students.  This  three-pronged  approach  provides  for  a  rich  and  broad  diversity  of 
scholarly  rigor  mixed  with  a  good  blend  of  practitioner  experience  in  the  field  of  acquisition. 
We  are  grateful  to  those  of  you  who  have  contributed  to  our  research  program  in  the  past 
and  hope  this  symposium  will  spark  even  more  participation. 

We  encourage  you  to  be  active  participants  at  the  symposium.  Indeed,  active 
participation  has  been  the  hallmark  of  previous  symposia.  We  purposely  limit  attendance  to 
350  people  to  encourage  just  that.  In  addition,  this  forum  is  unique  in  its  effort  to  bring 
scholars  and  practitioners  together  around  acquisition  research  that  is  both  relevant  in 
application  and  rigorous  in  method.  Seldom  will  you  get  the  opportunity  to  interact  with  so 
many  top  DoD  acquisition  officials  and  acquisition  researchers.  We  encourage  dialogue  both 
in  the  formal  panel  sessions  and  in  the  many  opportunities  we  make  available  at  meals, 
breaks,  and  the  day-ending  socials.  Many  of  our  researchers  use  these  occasions  to 
establish  new  teaming  arrangements  for  future  research  work.  In  the  words  of  one  senior 
government  official,  “I  would  not  miss  this  symposium  for  the  world  as  it  is  the  best  forum 
I’ve  found  for  catching  up  on  acquisition  issues  and  learning  from  the  great  presenters.” 

We  expect  affordability  to  be  a  major  focus  at  this  year’s  event.  It  is  a  central  tenet  of 
the  DoD’s  Better  Buying  Power  initiatives,  and  budget  projections  indicate  it  will  continue  to 
be  important  as  the  nation  works  its  way  out  of  the  recession.  This  suggests  that  research 
with  a  focus  on  affordability  will  be  of  great  interest  to  the  DoD  leadership  in  the  year  to 
come.  Whether  you’re  a  practitioner  or  scholar,  we  invite  you  to  participate  in  that  research. 

We  gratefully  acknowledge  the  ongoing  support  and  leadership  of  our  sponsors, 
whose  foresight  and  vision  have  assured  the  continuing  success  of  the  ARP: 
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Logistics) 

•  Director,  Acquisition  Career  Management,  ASN  (RD&A) 

•  Program  Executive  Officer,  SHIPS 

•  Commander,  Naval  Sea  Systems  Command 

•  Program  Executive  Officer,  Integrated  Warfare  Systems 

•  Army  Contracting  Command,  U.S.  Army  Materiel  Command 
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•  Office  of  the  Assistant  Secretary  of  the  Air  Force  (Acquisition) 

•  Office  of  the  Assistant  Secretary  of  the  Army  (Acquisition,  Logistics,  & 
Technology) 

•  Deputy  Director,  Acquisition  Career  Management,  U.S.  Army 

•  Office  of  Procurement  and  Assistance  Management  Headquarters, 
Department  of  Energy 

•  Director,  Defense  Security  Cooperation  Agency 

•  Deputy  Assistant  Secretary  of  the  Navy,  Research,  Development,  Test  & 
Evaluation 

•  Program  Executive  Officer,  Tactical  Aircraft 

•  Director,  Office  of  Small  Business  Programs,  Department  of  the  Navy 

•  Director,  Office  of  Acquisition  Resources  and  Analysis  (ARA) 
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Panel  20.  Application  of  an  App  Store  Software 
Model  Within  the  DoD 


Thursday,  May  17,  2012 

1:45  p.m.  - 
3:15  p.m. 

Chair:  Brigadier  General  Michael  E.  Williamson,  USA,  Joint  Program  Executive 
Officer,  Joint  Tactical  Radio  System 

Joint  and  Coalition  Tactical  Networking:  There’s  an  App  for  That!  Improving 
Affordability  and  Accelerating  Innovation  in  Tactical  Networking  Using  the 

Joint  Tactical  Radio  System  Enterprise  Business  Model 

Jeffery  Hoyle,  Joint  Tactical  Radio  System 

Widget  and  Mobile  Technologies  a  Forcing  Function  for  Acquisition  Change: 
Paradigm  Shift  Without  Leaving  Bodies  Behind 

Michael  Morris,  Christopher  Raney,  Kenneth  Trabue,  Timothy  Boyce,  Kari 

Nip,  Space  and  Naval  Warfare  Systems  Center  Pacific 

Apple  App  Store  as  a  Business  Model  Supporting  U.S.  Navy  Requirements 

Douglas  Brinkley  and  Brad  Naegle 

Naval  Postgraduate  School 

Michael  E.  Williamson — General  Williamson  assumed  his  duties  as  joint  program  executive  officer 
for  the  Joint  Tactical  Radio  System  in  March  201 1 . 

General  Williamson  was  born  in  Tucson,  AZ.  He  was  commissioned  at  the  University  of  Maine  as 
a  second  lieutenant  in  the  Air  Defense  Artillery  in  1983. 

His  assignments  include  service  as  the  automation  officer  for  the  32nd  AADCOM  in  Darmstadt, 
Germany.  He  then  served  as  a  chaparral  platoon  leader,  vulcan  platoon  leader,  maintenance  officer, 
and  executive  officer  in  C  Battery,  108th  Brigade,  Hahn  Air  Force  Base,  Germany.  After  attending  the 
Air  Defense  Artillery  Advance  Course,  he  served  as  the  chief,  Forward  Area  Air  Defense  Weapons, 
Development  Branch  at  Fort  Bliss,  TX.  He  then  commanded  B  Battery,  3/1  ADA  (Hawk)  in  the  1 1th 
Brigade  at  Fort  Bliss  and  also  in  the  31st  ADA  Brigade  at  Fort  Hood,  TX.  After  completing  command, 
he  served  as  the  Assistant  S-3  in  the  31st  ADA  Brigade. 

His  acquisition  experience  began  as  senior  military  software  analyst  at  NATO’s  military 
headquarters  in  Mons,  Belgium.  He  then  served  as  the  associate  director,  Battle  Command  Battle 
Lab  at  Fort  Leavenworth,  KS.  After  attending  Command  and  General  Staff  College,  he  served  as  the 
chief  of  information  technology,  Acquisition  Career  Management,  within  the  Office  of  the  Assistant 
Secretary  of  the  Army  for  Acquisition,  Logistics,  and  Technology.  He  was  then  selected  as  a 
congressional  fellow  and  served  as  a  legislative  assistant  to  a  member  of  Congress.  After  completing 
the  fellowship,  General  Williamson  served  as  the  product  manager  for  the  Global  Command  and 
Control  System-Army,  and  then  as  the  acquisition  military  assistant  to  the  Secretary  of  the  Army.  He 
served  as  commander  of  the  Software  Engineering  Center-Belvoir  (SEC-B).  He  was  then  assigned  as 
the  project  manager,  Future  Combat  System  (Brigade  Combat  Team)  Network  Systems’  Integration 
within  program  manager,  Future  Combat  System  (Brigade  Combat  Team).  He  then  served  as  the 
director  of  systems  integration,  within  the  Office  of  the  Assistant  Secretary  of  the  Army  for  Acquisition, 
Logistics,  and  Technology.  Prior  to  his  current  assignment,  General  Williamson  served  as  the  deputy 
program  manager,  Program  Executive  Office,  Integration. 

General  Williamson’s  awards  and  decorations  include  the  Legion  of  Merit  with  two  Oak  Leaf 
Clusters;  the  Meritorious  Service  Medal  with  two  Oak  Leaf  Clusters;  the  Joint  Service  Commendation 
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Medal,  the  Army  Commendation  Medal  with  two  Oak  Leaf  Clusters,  the  Joint  Service  Achievement 
Medal,  the  Army  Achievement  Medal  with  two  Oak  Leaf  Clusters,  the  Army  Superior  Unit  Award,  the 
National  Defense  Service  Medal  with  Bronze  Star,  the  Global  War  on  Terrorism  Service  Ribbon,  the 
Army  Service  Ribbon,  the  Overseas  Ribbon,  and  the  Army  Staff  Identification  Badge. 

General  Williamson’s  education  includes  a  Bachelor  of  Science  from  Husson  College  in  business 
administration,  a  Master  of  Science  in  systems  management  from  the  Naval  Postgraduate  School, 
and  a  PhD  in  business  administration  from  Madison  University.  He  also  has  graduate  certificates  in 
public  policy  from  the  JFK  School  of  Government,  Harvard  University,  and  the  Government  Affairs 
Institute  at  Georgetown  University.  He  is  a  graduate  of  the  Army  Command  and  General  Staff 
College,  a  graduate  of  the  Advanced  Management  Program  at  the  Harvard  Business  School,  and 
was  a  Senior  Service  College  Fellow  at  the  University  of  Texas  at  Austin.  He  is  Level  III  certified  in 
program  management  and  communications  and  computers. 
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Widget  and  Mobile  Technologies  a  Forcing  Function  for 
Acquisition  Change:  Paradigm  Shift  Without  Leaving 

Bodies  Behind 


Michael  Morris — Morris  has  29  years  of  experience  in  C4I  systems,  including  most  recently  working 
as  lead  systems  engineer  (LSE)  for  DoD/PEO  C4I  Marketplace  efforts.  He  previously  served  as 
deputy  project  manager/LSE  for  the  Collaborative  Force  Analysis  Sustainment  and  Transportation  at 
SSCPAC.  Morris  has  also  worked  in  the  Test  and  Evaluation  of  Net-Enabled  Command  and  Control, 
gaining  extensive  knowledge  in  service-oriented  architectures.  Additionally,  he  has  extensive 
experience  in  Global  Command  and  Control  System-Maritime/Joint  test  plan  development/procedures 
and  reports.  As  COMSPAWARSYSCOM  04F  Afloat  test  director  (2000-2004),  he  was  responsible  for 
testing  all  SPAWAR  products  installed  on  U.S.  Naval  vessels,  [michael.a.morris4@navy.mil] 

Christopher  Raney — Raney  serves  as  the  head  of  the  Network-Centric  Command  Systems  Branch 
at  SSCPAC.  A  native  of  San  Diego,  CA,  Raney  graduated  from  San  Diego  State  University  (SDSU), 
summa  cum  laude,  with  a  Bachelor  and  a  Master  of  Science  degree  in  computer  science.  Raney 
received  the  prestigious  Outstanding  Computer  Science  Graduate  award,  given  to  the  top  graduating 
computer  science  student  at  SDSU  each  year.  Raney  subsequently  joined  SSCPAC  as  a  computer 
scientist.  He  has  worked  in  a  broad  spectrum  of  areas,  including  net-centric  systems,  information 
assurance,  cyber  warfare,  cloud  computing,  enterprise  networks,  communications,  and  command  and 
control,  [raneyc@spawar.navy.mil] 

Kenneth  Trabue — Trabue  is  the  project  manager  for  the  Global  Command  and  Control  System — 
Joint,  Integrated  Imagery  and  Intelligence  (GCCS-J  13)  program  at  SSCPAC.  In  addition  to  his 
program  management  responsibilities,  he  also  leads  the  GCCS-J  13  widget  development  effort  under 
the  Joint  C2  Common  User  Interface  (JC2CUI)  project.  Trabue  is  a  retired  U.S.  Coast  Guard  C4ISR 
officer  who  served  as  the  IT  systems  branch  head  for  the  USCG  Intelligence  Coordination  Center, 
and  was  the  USCG  program  manager  for  the  CG  Intelligence  Support  System.  He  has  a  Bachelor  of 
Science  degree  in  computer  networking,  [kenneth.trabue@navy.mil] 

Timothy  Boyce — Boyce  has  over  24  years  of  experience  in  C4I  systems  and  serves  as  the  head  of 
the  Strike  Planning  and  Execution  Branch  at  SSCPAC.  He  also  provides  project  management  support 
to  the  Global  Command  and  Control  System — Joint,  Integrated  Imagery  and  Intelligence  (GCCS  -  J 
13)  program.  Prior  to  his  selection  as  a  branch  head,  Boyce  was  the  assistant  program  manager  for 
logistics  for  Naval  Mission  Planning  Systems  at  the  Strike  Planning  and  Execution  Systems  Program 
Office  (PMA-281).  Boyce  is  certified  DAWIA  Level  III  in  Information  Technology  and  Level  II  in 
Program  Management  and  Lifecycle  Logistics,  [timothy.boyce@navy.mil] 

Kari  Nip — Nip  graduated  from  the  University  of  California,  San  Diego,  with  a  Bachelor  of  Science  in 
electrical  engineering.  She  has  since  served  as  an  electronics  engineer  in  the  Information  Warfare 
Branch  at  SSCPAC.  She  has  provided  engineering  and  technical  support  to  the  DoD  Storefront  and 
PEO  C4I  Marketplace  efforts.  Prior  to  her  work  on  these  projects,  she  served  as  a  systems  engineer 
for  the  SBINet  Demonstration  project  involving  RF  communications  and  networked  sensors. 
[kari.nip@navy.mil] 

Abstract 

The  Department  of  Defense  (DoD)  software  acquisition  policy  struggles  to  adapt  to  the 
emerging  trend  of  delivering  lightweight  applications  on  demand  via  application  store 
technologies.  The  commercial  world  has  evolved  to  where  it  now  provides  a  constant  stream 
of  capabilities  allowing  customers  to  customize  their  information/communication  devices  with 
numerous  applications. 

DoD-created  solutions  are  often  inflexibly  designed  with  limited  adaptability  due  to 
requirements  for  reliability,  communications  security,  and  absolute  need  for  accuracy. 
Because  of  this,  acquisition  processes  are  needed  that  allow  warfighters  to  take  advantage  of 
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the  rich  Internet  applications  and  Web  2.0  technologies  currently  available  to  the  average 
consumer. 

This  paper  identifies  processes  for  employing  a  DoD  application  store  that  delivers  software 
in  a  rapid,  secure,  and  reliable  manner.  Leveraging  the  government-developed  open  source 
Ozone  Widget  Framework,  web  applications  are  developed  and  registered  to  a  single 
repository.  The  warfighter  can  discover,  access,  and  compose  these  web  applications  from 
the  Ozone  Marketplace. 

To  promote  innovation  and  foster  collaboration  between  the  DoD  and  industry,  the  application 
stores  in  the  DoD  should  include  separate  industry  trial  and  demonstration  sections.  Industry 
partners  can  upload  capabilities  for  demonstration/trial  by  the  government  and  allow  program 
offices  to  evaluate  applications  with  the  goal  of  selecting  capabilities  for  Programs  of  Record. 

Introduction 

The  Department  of  Defense  (DoD)  struggles  to  keep  up  with  the  commercial  world 
when  delivering  new  software  technology  to  its  customers.  By  the  time  new  solutions  are 
deployed  to  the  warfighter,  the  technology  is  obsolete.  The  software  industry’s  focus  has 
evolved  to  include  small  flexible  mobile  code  via  widgets  and  other  mobile  applications  (also 
referred  to  as  “apps”).  Several  programs  within  the  DoD  have  started  similar  initiatives  that 
hold  the  promise  of  reducing  the  “heavy  lifting”  required  as  part  of  the  current  acquisition 
process. 

IBM’s  2006  Global  Technology  Outlook  recognized  the  importance  of  a  rapidly 
evolving  software  development  paradigm  as  a  driving  force  in  web-based  dynamic  content 
and  the  manner  in  which  it  is  delivered  to  the  user  primarily  through  “situational 
applications.” 

Software  development  is  going  through  a  rapid  evolution  enabled  by  the 
ubiquity  and  ease-of-use  of  the  web,  simple  to  use  software,  tools,  and 
techniques,  dramatic  rise  in  computer  literacy,  and  the  development  of 
standards  around  Web  Services.  All  these  forces  together  are  giving  rise  to  a 
new  paradigm  for  the  collaboration,  creation,  manipulation  of  dynamic  content 
with  the  web  as  the  platform,  a.k.a.  Web  2.0.  The  building  of  situational 
applications  -  applications  built  with  just  enough  function  to  satisfy  a  business 
need,  usually  by  business  users  -  by  mixing  and  re-mixing  existing 
components  are  becoming  more  and  more  common.  These  trends  will  force 
businesses  to  rethink  how  their  applications  and  services  are  designed, 
developed,  and  managed.  This  in  turn  will  put  the  onus  on  IT  infrastructure 
companies  to  offer  new  tools  for  development,  management  and  integration 
of  situational  applications  and  services.  (IBM  Research,  2006)1 

IBM’s  insight  into  the  emerging  environment  of  situational  applications  and  dynamic 
content  demonstrated  their  predictive  ability  to  understand  forces  that  would  significantly 
impact  software  development  trends  from  six  years  ago.  Since  that  time,  the  creation  and 
use  of  situational  applications  throughout  the  private  sector  has  exploded.  While  the  DoD 
has  only  started  to  make  inroads  within  this  environment,  several  Programs  of  Record 
(POR)  have  embraced  widgets  and  other  mobile  technologies,  hoping  to  enhance  warfighter 
situational  awareness  and  access  to  information.  Unfortunately,  the  Defense  Acquisition 
System  has  not  adapted  to  this  new  environment,  making  it  difficult  to  field  these 
technologies  rapidly  to  meet  emergent  requirements. 


1  Courtesy  of  International  Business  Machines  Corporation,  ©  2006  International  Business  Machines 
Corporation. 
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Problem  Statement — Defense  Acquisition  System 

The  2010  findings  and  recommendations  report  from  the  House  Armed  Services 
Committee  Panel  on  Defense  Acquisition  Reform  (Andrews  et  al.,  2010)  cited  ongoing 
problems  with  the  performance  of  weapons  systems  acquisition  programs  throughout  the 
DoD,  noting  $296  billion  in  cost  growth  and  an  average  schedule  delay  of  22  months  (p.  6). 
While  the  report  specifically  criticizes  Major  Defense  Acquisition  Programs  (MDAP),  given 
the  software-intensive  development  associated  with  many  MDAPs,  they  are  a  valuable 
benchmark  in  understanding  where  issues  and  concerns  remain  within  the  Defense 
Acquisition  System.  The  report  further  notes  issues  with  excessive  development  times, 

“Even  in  the  Department’s  ninth  year  of  active  warfare  during  which  large  quantities  of 
equipment  have  been  consumed  and  numerous  new  mission  needs  have  been  generated, 
weapon  systems  acquisition  remains  typified  by  programs  with  development  timelines 
lasting  more  than  a  decade”  (Andrews  et  al.,  2010,  p.  6).  According  to  the  panel  report,  the 
problem  is  worse  when  compared  with  private-sector  development  and  update  cycles. 
Where  private-sector  cycles  are  12-18  months,  defense  IT  systems  can  routinely  require 
48-60  months  to  deliver  a  capability  (Andrews  et  al.,  2010,  p.  17). 

The  problem  with  the  defense  acquisition  process  is  well  known  within  the  DoD. 
According  to  a  2008  GAO  report  on  defense  acquisitions  (Schinasi,  2008), 

A  senior  Army  acquisition  official  recently  testified  before  Congress  that 
because  the  process  can  take  more  than  a  year,  it  is  not  suitable  for  meeting 
urgent  needs  related  to  ongoing  operations;  and  a  recent  study  by  the  Center 
for  Strategic  and  International  Studies  indicates  that  the  process  is  unwieldy 
and  officials  are  now  trying  to  find  ways  to  work  around  it.  (p.  6) 

Given  the  listed  problems  with  current  acquisitions  regarding  cost  overruns  and 
schedule  delays,  it  is  easy  to  see  that  the  flexibility  and  speed  necessary  to  meet  the  needs 
of  today’s  warfighter  is  not  a  critical  factor  within  the  Defense  Acquisition  System. 

Details  of  the  Current  Process 

In  the  current  acquisition  process,  there  are  long  lead  times  from  development  to 
deployment  due  to  the  need  for  highly  secure,  accurate,  and  reliable  products. 

The  Report  of  the  Committee  on  Armed  Services,  House  of  Representatives  on  H.R. 
5122  (U.S.  Congress,  2006),  revealed  that  MDAPs  within  the  DoD  have  continued  to  exhibit 
increasing  costs  and  extended  schedules,  which  have  a  detrimental  impact  on  the  DoD’s 
ability  to  field  systems: 

The  rising  cost  and  lengthening  production  schedules  of  major  defense 
acquisition  programs  has  led  to  more  expensive  platforms  fielded  in  fewer 
numbers.  The  committee  believes  that  internal  DOD  pressure  to  develop 
follow-on  weapons  systems  that  include  all  necessary  and  anticipated  military 
capabilities  may  create  an  over-reliance  on  individual  “mega”  systems  that 
are  potentially  more  expensive  and  time-consuming  to  develop  than  less 
sophisticated  but  capable  systems,  (p.  15) 

Warfighters’  emerging  requirements  cannot  be  met  by  the  lengthy  cradle-to-grave 
monolithic  super  systems  of  today  and  the  past.  Current  trends  toward  lightweight  web  and 
mobile  applications  in  the  commercial  environment  have  encouraged  the  DoD’s 
development  of  these  technologies. 

Systems  of  record  are  currently  exploring  the  advantages  of  deploying  capabilities  in 
this  fashion;  the  long  pole  in  this  trend  is  the  restriction  applied  by  the  acquisition  rules  and 
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structure.  These  limitations  bog  down  the  creative  methods  for  exposing  these  lightweight 
capabilities.  The  added  burden  to  explore  various  hardware  platforms,  desktops,  laptops, 
tablets,  and  phones  presents  unique  issues  beyond  just  security.  The  efforts  to  provide 
oversight  and  governance  in  the  development  and  deployment  of  the  software  and  hardware 
packages  are  in  the  early  stages  and  are  being  worked  as  an  Integrated  Project  Team  (IPT) 
with  representation  from  key  stakeholders  across  the  DoD. 

The  process  of  running  a  widget  through  the  same  acquisition  milestones  as  a 
Program  of  Record  (POR)  is  cost  and  time  prohibitive.  The  time  and  money  that  would  be 
required  in  this  situation  removes  the  widget  from  a  warfighter’s  arsenal,  potentially 
answering  a  current  need,  and  relegates  it  to  the  acquisition  world  of  “someday.”  Cyber 
systems  represent  one  of  our  strongest  and  most  potent  weapons  and  are  critical  elements 
in  the  battle  space  of  warfare.  We  cannot  continue  to  equip  Soldiers  and  Sailors  with 
capabilities  that  are  inferior  or  unequal  to  those  of  our  adversaries. 

Commercial-Industry  Approach 

Current  acquisition  rules  applied  to  POR  systems  have  caused  a  major  gap  between 
the  technology  available  to  the  warfighter  and  that  which  is  commonly  used  within  the 
commercial  sector.  This  is  exacerbated  by  the  increased  capabilities  of  mobile  computing 
with  smart  phones  and  tablets.  The  average  person  now  possesses  more  computing  power 
than  some  mainstream  systems  used  in  our  military.  This  technology  gap  is  exaggerated  for 
younger  Service  members  exposed  and  accustomed  to  this  technology  and  then  forced  to 
use  outdated  information  technologies. 

Today’s  commercial  industry  provides  a  constant  stream  of  technological  capabilities 
through  an  almost  endless  supply  of  applications.  Customers  have  the  ability  to  search  for 
mobile  applications  that  function  to  fulfill  their  specific  needs,  and  they  are  also  provided 
continual  updates  to  existing  applications  that  offer  additional  functionality  and/or  increase 
performance.  This  allows  industry  leaders  to  provide  a  means  for  application  developers  to 
rapidly  make  their  applications  available  to  users  with  the  appropriate  communication 
device. 


Apple  Inc.,  a  leader  in  mobile  technology,  produces  devices  with  application  software 
capabilities.  The  Apple  App  Store  contains  Apple-created  applications  and  third-party  mobile 
applications,  which  are  available  where  they  may  be  discovered  and  downloaded  by  the 
customer.2  An  app  approval  process  is  in  place  to  ensure  that  submitted  applications  are 
reliable,  perform  as  explained,  and  adhere  to  Apple’s  stringent  requirements  regarding 
appropriate  content  (Apple  Inc.,  2012).  Developers  are  equipped  with  the  App  Store  Review 
Guidelines,  a  document  with  review  criteria,  rules,  and  examples  for  a  wide  range  of 
development  topics.  Upon  submission  for  review,  the  applications  are  checked  for 
compliance  with  the  App  Store  Review  Guidelines.  If  the  apps  comply  with  the  technical, 
content,  and  design  criteria,  they  are  made  available  in  the  App  Store.  If  an  application  is 
rejected,  the  developer  can  submit  an  appeal  to  the  App  Review  Board  if  it  is  believed  that 
the  functional  or  technical  implementation  was  misunderstood  (Apple  Inc.,  2012). 

In  the  Android  world,  registered  developers  of  applications  for  Google’s  Android 
Operating  System  (OS)  can  publish  their  application  to  the  Android  App  Section  of  Google 
Play,  provided  it  meets  a  set  of  requirements  that  are  enforced  by  the  Google  Play  server 
during  upload  of  the  application  (Google  Inc.,  2012).  Google  does  not  take  on  any  obligation 
to  monitor  applications  uploaded  to  Google  Play,  but  it  does  place  restrictions  on  application 


2  Widget  &  Mobile  Technologies  a  Forcing  Function  for  Acquisition  Change  is  an  independent  research  paper 
and  has  not  been  authorized,  sponsored,  or  otherwise  approved  by  Apple  Inc. 
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content,  network  usage,  spam,  and  processing  payments.  If  an  application  violates  these 
restrictions,  Google  has  the  authority  to  remove  the  application  from  Google  Play  (Google 
Inc.,  2012).  Without  a  review  process,  it  may  only  take  a  couple  minutes  to  an  hour  to 
publish  an  application  to  Google  Play  and  make  it  available  to  Android  device  users. 

The  Amazon  Appstore,  an  alternative  mobile  application  store  for  Google’s  Android 
OS,  also  has  an  approval  process  for  apps.  Apps  submitted  to  the  Amazon  Appstore  are 
tested  to  ensure  that  they  work  as  described  in  the  product  description,  do  not  impair  the 
functionality  of  the  user’s  mobile  device,  and  comply  with  the  Appstore  Distribution 
Agreement  and  Amazon’s  content  guidelines.  If  reviewers  have  a  question  about  the  app  or 
if  the  app  does  not  meet  the  Amazon  Appstore’s  acceptance  criteria,  the  submitter  is  notified 
via  e-mail  (Amazon.com  Inc.,  2012). 

Overview  of  Recommended  Solution 

The  Department  of  the  Navy  (DoN)/DoD  lacks  a  lightweight  web  application  test  and 
integration  environment  to  effectively  model,  test,  exercise,  and  perform  certification  and 
accreditation  of  widget  capabilities.  Although  the  need  for  this  environment  has  been 
recognized,  there  is  a  lack  of  dedicated  funding  for  an  infrastructure-supported  technological 
solution.  This  environment  is  essential  for  the  DoN  workforce  to  effectively  meet  widget 
operational  missions  through  individual  developer  testing.  Additionally,  if  the  DoN  is  to 
realize  efficient  and  effective  deployments  of  new,  highly  accurate,  secure,  and  functional 
technologies,  a  comprehensive  test  and  integration  (T&l)  and  evaluation  environment  is 
required. 

The  requirement  for  a  widget  T&l  environment  has  been  identified  with  widget 
technology  development  throughout  the  DoD.  Current  widget  efforts  include  DoD  Chief 
Information  Officer  (  CIO)  Storefront  Pilot,  National  Security  Agency  (NSA)  Denver  Store, 

Global  Command  and  Control  Systems — Joint  Integrated,  Imagery  and  Intelligence  (GCCS- 
J  I3),  Command  and  Control  Rapid  Prototyping  Continuum  (C2RPC),  Integrated  Intelligence 
Architecture  (I2A),  and  so  forth. 

This  document  proposes  a  DoN/DoD  widget  T&l  environment  that  incorporates  the 
unique  and  common  aspects  of  Navy  widget  environments.  This  T&l  environment  will 
replicate  not  only  the  environment  that  connects  directly  to  the  Global  Information  Grid 
(GIG),  but  also  Navy  afloat,  ashore,  deployed,  and  limited  communication  environments 
(hardware,  software,  and  associated  configurations).  The  resulting  environment  will  be 
separate  from  operational  environments  and  provide  the  DoN  the  capability  to  perform  T&l, 
as  well  as  automated  testing  to  include  certification  and  accreditation  (C&A)  of  widgets.  This 
will  create  a  Navy-trusted  source  environment  for  widget  transition  to  the  fleet  and  develop 
more  effective  and  efficient  techniques,  processes,  and  procedures  to  achieve  speed  to 
capability. 

The  proliferation  of  widget  technology  requires  standards,  specifications,  processes, 
and  a  T&l  environment  to  ensure  proper  products  are  discoverable  by  the  warfighter.  The 
widget  T&l  environment  will  provide  a  composable  mission-based  set  of  software 
capabilities  to  accomplish  mission  tasking  and  will  enable  software-based  mission 
capabilities  to  be  released  in  a  couple  weeks  rather  than  months  or  years. 

A  DoD  marketplace  or  application  store  could  rapidly  and  securely  field  software 
capabilities  to  the  warfighter  in  the  form  of  widgets  and  web  applications.  Widgets  provide  a 
technological  capability  to  foster  this  rapid  fielding  ability.  Widgets  continue  to  gain  in 
popularity  over  traditional  thick  applications  due  to  their  ease  of  creation,  variety  of 
capabilities,  and  reduced  development  time  (MITRE  Corporation,  2012).  Since  widgets 
provide  very  specific  functionality  and  generally  require  a  relatively  small  amount  of  software 
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code,  developers  can  create  them  on  a  relatively  short  timeline.  This  means  new  capabilities 
can  be  developed  quickly  in  response  to  user  needs  and,  with  a  widget  T&l  environment  and 
an  application  store,  these  widget  capabilities  can  be  made  available  for  search  and 
discovery  by  the  user. 

Detailed  Architecture  and  Process 

Ozone  Widget  Framework  (OWF) 

What  is  a  Widget? 

The  Ozone  Widget  Framework  User’s  Guide  (Boyd,  2011)  refers  to  widgets  as 
lightweight,  single-purpose,  web-enabled  applications  that  users  can  configure  to  their 
specific  needs.  Widgets  can  provide  summary  information  or  a  limited  view  into  a  larger 
application.  They  can  also  be  used  alongside  related  widgets  to  provide  an  integrated  view 
as  required  by  the  user  (Boyd,  201 1 ). 

Ozone  Widget  Framework 

The  Ozone  Widget  Framework  (OWF)  is  a  platform  that  offers  infrastructure  services 
to  simplify  the  development  of  workflows  and  presentation-tier  application  integration.  It  is 
also  a  layout  manager  for  the  operation  of  widgets  on  a  single  web  page.  Widgets,  which 
are  web  applications  that  can  be  installed  and  executed  in  a  web  browser,  display 
information  or  provide  dynamic  content  from  a  backend  or  local  service.  Just  like  any  widget 
framework,  the  OWF  supplies  the  structure  and  templates  for  creating  widgets,  providing 
users  with  the  capability  to  develop,  share,  and  operate  widgets.  Unlike  a  standard  browser 
window,  the  OWF  allows  users  to  load  and  operate  multiple  widgets  within  a  single  webpage 
rather  than  opening  multiple  browser  windows  or  tabs  to  display  more  than  one  widget.  This 
allows  users  to  view  a  great  amount  of  information  on  a  single  webpage.  From  an 
intelligence  analyst’s  standpoint,  the  OWF  provides  a  means  to  conveniently  search, 
access,  and  display  intelligence  data  on  a  single  display. 

The  OWF  allows  users  to  load  widgets,  select  a  layout  type  called  a  dashboard 
layout,  and  customize  the  arrangement  of  the  widgets  within  the  dashboard.  The  OWF 
supports  multiple  dashboard  layouts,  including  desktop,  tabbed,  portal,  and  accordion.  The 
desktop  layout  allows  users  to  arrange  and  drag  widgets  anywhere  within  the  browser 
window,  much  like  a  desktop  application  on  a  standard  operating  system  desktop.  The 
tabbed,  portal,  and  accordion  layouts  fix  the  widget  positions  in  the  browser,  but  users  are 
able  to  select  which  widgets  are  assigned  to  the  fixed  locations,  creating  a  customized 
display.  The  dashboard  layout  and  arrangement  of  widgets  is  saved  when  a  user  logs  out  of 
the  OWF  so  the  next  time  the  account  is  accessed  the  entire  layout  is  maintained  (Boyd, 
2011). 

The  OWF  also  provides  a  suite  of  application  programming  interfaces  (APIs), 
allowing  widget  developers  to  enhance  their  web  applications  using  inter-widget 
communication,  user  preferences,  and  internationalization.  Each  API  is  written  in  JavaScript 
so  that  widgets  can  be  built  in  a  large  variety  of  web  technologies.  Therefore,  widgets  can 
be  written  in  the  JavaScript-capable  technology  of  the  developer’s  choice. 

The  OWF,  originally  developed  and  sponsored  by  the  NSA  as  a  government  off-the- 
shelf  (GOTS)  solution,  is  now  government  open-source  software  (GOSS)  with  a 
collaborative  software  development  model.  The  OWF  GOSS  Program  is  responsible  for  the 
maintenance  of  OWF  and  Ozone  Marketplace  (OMP)  software  releases.  The  OWF  GOSS 
board,  currently  comprised  of  members  from  NSA,  the  DoD  CIO,  Intelink,  SPAWAR 
Systems  Center  Pacific  (SSCPAC),  the  Central  Intelligence  Agency  (CIA),  the  Defense 
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Information  Systems  Agency  (DISA),  the  National  Reconnaissance  Office  (NRO),  and  the 
Intelligence  and  Security  Command  (INSCOM),  can  distribute  development  priorities  to  any 
government  agency  or  program  requesting  the  source  code  for  either  its  own  use  or  for 
updating.  These  agencies  are  encouraged  to  submit  software  patches  and  feature 
enhancements  to  improve  the  baseline  code  and  benefit  the  community  of  projects  utilizing 
the  OWF  and  OMP. 

Widgets  in  Action 

Command  and  Control  and  Intelligence  Widgets 

Several  communities  within  the  DoD  have  embraced  the  OWF  and  widgets.  For 
example,  the  GCCS-J  I3  program  has  been  actively  developing  widgets  for  various  Naval 
commands  (I3  Common  Geospatial  Display  Widget,  I3  Vessels  of  Interest  Widget,  I3 
Maneuver  Unit  Widget,  I3  Latest  DMOB  Equipment  Widget,  I3  Naval  Activity  Widget,  I3 
Channel  List  Widget,  I3  Blue  Forces  Widget,  l3AOB  Widget,  I3  Recent  Activity  Widget,  I3 
Targeting  Widget,  I3  Weather  Observation  Widget,  and  I3  Weather  Forecasting  Widget).  The 
Distributed  Common  Ground  System — Army  (DCGS-A)  has  created  a  suite  of  widgets  for 
their  users  (Common  Admin  Widget,  Common  Help  Widget,  Common  Query  Widget, 
Common  Map  Widget,  Coordinate  Conversion  Widget,  DIB  Query  Widget,  plus  over  50 
additional  widgets  providing  weather,  HUMINT,  IMINT,  Geospatial,  and  Alerting  tools).  The 
Defense  Intelligence  Information  Enterprise  (DI2E)  has  selected  OWF  for  use  within  its 
development  and  the  Joint  Command  and  Control  Common  User  Interface  (JC2CUI)  has 
selected  the  OWF  as  one  of  its  two  common  clients.  GCCS-J  13,  DI2E,  and  JC2CUI  have  all 
embraced  the  OWF  to  provide  situational  awareness  and  improved  usability  tools  to  their 
users,  but  each  program  requires  a  consolidated  resource  from  which  the  widgets  and 
associated  services  are  made  discoverable.  These  are  not  the  only  widget  development 
efforts  underway  within  the  DoD;  there  are  many  operational  commands  that  have 
expressed  interest  in  developing  their  own  widgets.  This  operationally  focused  development 
effort  is  affectionately  known  as  “Engineering  at  the  Edge.” 

Engineering  at  the  Edge 

The  DoD’s  dependence  on  large-scale  acquisition  programs  is  well  documented. 
MDAPs  that  provide  end-to-end  solutions  attempting  to  satisfy  a  system’s  full  requirements 
baseline  often  produce  very  complex  applications  that  are  difficult  to  use  and  train.  These 
projects  are  also  subject  to  strict  acquisition  and  other  approval,  testing,  and  fielding 
requirements.  Engineering  at  the  Edge  seeks  to  use  the  OWF  and  widgets  to  reduce  an 
operational  command’s  dependency  on  end-to-end  systems  development  efforts  that  need 
years  to  develop  and  may  not  address  requirements  that  have  emerged  since  development 
started.  Fortunately,  the  OWF  has  enabled  the  DoD  community  to  decouple  the  user 
interface  and  visualization  components  from  the  rest  of  the  architecture. 

Over  the  past  several  years,  many  of  today’s  PORs  have  developed  robust  data 
services  and  web-service  interfaces  to  support  third-party  client  data  access  requirements. 
Understanding  that  operational  commands  need  access  to  these  robust  data  services, 
Engineering  at  the  Edge  offers  additional  capability  by  supplementing  the  OWF  with  a  robust 
C4ISR  Widget  Developer’s  Toolkit  (WDT).  The  WDT  includes  widget  guidance,  training,  and 
a  functional  software  development  kit  that  will  allow  the  developer  community  to  be 
extended  to  include  the  operational  and  tactical  commands,  within  a  controlled  environment. 
These  Engineering  at  the  Edge  developers  will  now  have  the  tools  and  processes  necessary 
to  manage  and  view  data  based  on  specific  user  needs  in  order  to  meet  ad  hoc  and 
emerging  operational  requirements.  Using  the  WDT,  Engineering  at  the  Edge  developers 
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can  leverage  the  robust  C4ISR  architecture  that  has  been  deployed  by  large  programs, 
without  being  tied  to  their  rigid  cost,  performance,  and  schedule  restrictions. 

With  the  abundance  of  existing  widgets  and  potential  for  numerous  widgets  created 
through  Engineering  at  the  Edge,  users  need  a  common  resource  to  search  for  and  discover 
existing  widgets  for  use.  This  is  accomplished  through  the  use  of  the  Ozone  Marketplace. 

Ozone  Market  Place  (OMP) 

The  Ozone  Marketplace  (OMP)  is  a  thin-client  registry  of  applications  and  services 
similar  to  a  commercial-industry  application  store,  such  as  the  Apple  App  Store  or  Google 
Play.  Generally,  it  is  a  directory  where  widgets  are  submitted  and  can  be  shared  for  others 
to  search,  access,  and  use.  The  OMP  is  the  marketplace  specific  to  the  OWF  (Figure  1).  It 
can  also  stand  on  its  own  but  is  usually  utilized  with  Ozone.  The  OMP  is  also  a  part  of  the 
OWF  GOSS  Program  so  it  undergoes  updates  and  new  releases  made  by  the  OWF  GOSS 
Board. 


From  a  user  standpoint,  the  OMP  is  where  analysts  can  search  for  widgets  that 
provide  desired  information  and  can  add  the  selected  widgets  to  their  system  for  use. 
Developers  can  upload  their  widgets  to  the  OMP  and  provide  associated  metadata,  but 
OMP  administrators  have  the  ability  to  approve  or  reject  widgets  submitted  to  the  OMP. 
Therefore,  users  can  only  utilize  widgets  once  they  have  been  approved  by  the  OMP 
administrators.  One  example  of  an  OMP  for  warfighter  use  is  the  PEO  C4I  Marketplace. 


Ozone  Widget  Framework 


User’s  Personalized 
Workspace 


Search  for  Widgets 
to  add  to  Personal 
Workspace 


Add  Widgets  to  Marketplace 


Figure  1.  Ozone  Widget  Framework/Ozone  Marketplace 
PEO  C4I  Marketplace  Overview 

Before  new  capabilities  are  made  available  to  the  warfighter,  they  must  undergo 
developmental  tests,  operational  tests,  and  a  strict  C&A  process.  All  of  these  tests  can  take 
as  long  as  nine  months,  enough  time  for  the  “new”  technology  to  become  out  of  date  and 
unresponsive  to  immediate  user  needs.  One  of  PEO  C4l’s  FY2012  strategic  goals  is  to 
“foster  focused  innovation  to  rapidly  field  relevant  capabilities  to  meet  existing  and  emerging 
warfighter  needs”  (Goal  2.4).  Widgets  provide  a  technological  capability  to  foster  this  rapid 
fielding  ability  and  provide  the  potential  to  rapidly  implement  Command,  Control, 
Communications,  Computers,  Intelligence,  Surveillance  and  Reconnaissance  (C4ISR)  and 
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operational  capabilities  to  the  warfighter.  Widgets  are  being  deployed  in  the  Navy 
operational  environment  as  part  of  formal  software  builds  and  releases  for  PORs.  However, 
the  traditional  method  of  providing  software  to  the  fleet  typically  does  not  support  agile 
deployment  of  widgets. 

To  further  Goal  2.4  in  August  201 1 ,  the  following  two  tasks  were  approved  by  PEO 

C4I: 


•  Task  2.4.1 — Set  up  an  application  storefront  on  SIPRNET  and  JWICS  for  the 
delivery  of  C4I  widgets. 

•  Task  2.4.2 — Establish  an  Agile  Widget  Approval  IPT  to  develop  a  business 
process  for  developing,  modifying,  approving,  and  remotely  deploying 
widgets. 

The  PEO  C4I  Marketplace  and  a  governance  process  specific  to  widgets  submitted 
by  an  accredited  POR  will  reduce  lead  times  and  ensure  that  widgets  are  efficiently  and 
securely  introduced  in  a  production  environment  for  the  warfighter. 

PEO  C4I  Marketplace  architecture  can  decrease  the  infrastructure,  and  certification 
and  accreditation  burden  on  the  operational  user  by  decoupling  the  widget  capabilities  from 
his  or  her  browser  in  the  operational  environment.  Figure  2  depicts  the  operations 
architecture  of  the  PEO  C4I  Marketplace.  An  operational  user  can  discover  widget 
capabilities  from  metadata  in  his  or  her  operational  Ozone  Marketplace,  which  are  then 
served  from  an  accredited  Ozone  Widget  Framework  server  to  accredited  Integrated 
Shipboard  Network  System  (ISNS)  devices  (e.g.,  desktops  or  mobile  devices).  The  widget 
may  actually  be  hosted  in  a  distinct  environment,  such  as  CANES  or  GCCS-I3,  which  can 
provide  the  backend  services  and  data  that  comprise  the  capability.  Because  a  widget, 
backend  services,  and  associated  data  may  reside  and  operate  completely  within  accredited 
environments  and  are  transported  over  secure  communications  means,  the  accreditation 
burden  can  be  greatly  reduced. 


Operational  User 


ISNS 

ATO 


Widget  Hosting  / 
Backend  Services/Data 


GCCS-I 3  Server 


Figure  2.  PEO  C4I  Marketplace  Operations  Architecture 

Other  PEO  C4I  efforts  to  quickly  deploy  new  technologies  to  the  warfighter,  such  as 
widget  development,  migration  of  PEO  C4I  capabilities  to  the  Cloud,  and  Cloud  Task  Force, 
will  be  brought  together  by  the  PEO  C4I  Marketplace  and  widget  governance  processes. 
They  demonstrate  a  unified  end-to-end  process  for  taking  a  widget  capability  through 
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development,  test,  certification,  approval,  and  delivery.  Figure  3  illustrates  the  integration  of 
the  PEO  C4I  Marketplace  and  the  Navy  Cloud. 
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Data  Cloud 
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PEO  C4I  Marketplace 
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Widgets,  Application,  Services 


Widget  &  Apps 
T&I  Environment 
Agile  Widget/App 
Approval 


Video 


Enterprise 

Cloud 


Storage  Cloud 


Afloat 


Figure  3.  PEO  C4I  Marketplace  Operations  Architecture 

The  PEO  C4I  Marketplace  seeks  to  increase  the  speed  at  which  new  capabilities  are 
provided  to  the  warfighter  by  creating  an  efficient  test,  verification,  and  validation  process  to 
govern  widgets.  Figure  4  depicts  the  operational  concept  of  the  PEO  C4I  Marketplace.  A 
widget  developer  produces  a  widget,  which  is  submitted  to  the  T&I  Marketplace 
Environment  for  testing.  The  PEO  C4I  Widget  T&I  Team  provides  feedback  to  the  widget 
developer  on  improvements  needed  to  make  the  widget  compliant  with  the  Operational  PEO 
C4I  Marketplace  standards,  enforced  by  the  Operational  Designated  Approval  Authority 
(ODAA)  and  Commander,  Operational  Test  and  Evaluation  (COMOPTEV).  Upon  completion 
of  all  testing,  the  widget  is  promoted  to  the  Operational  Marketplace  Environment.  From 
there,  the  operational  user  can  discover  the  widget  from  a  marketplace  (applications  store) 
and  consume  the  capability  in  an  operational  environment.  Ultimately,  the  operational  user 
can  provide  feedback  about  the  widget  to  build  on  the  existing  capability  or  to  inspire  new 
capabilities. 
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Figure  4.  PEO  C4I  Marketplace  Operational  Concept 


Widget  Governance  Tool 

Widget  governance  is  how  an  organization  establishes  and  controls  its  processes 
and  policies  regarding  widgets.  It  includes  a  system  to  track  and  record  where  a  widget  is 
within  a  widget  process  and  checks  for  its  compliance  with  existing  policies.  By  establishing 
an  efficient  test  and  evaluation  process  to  govern  widgets  and  approve  their  acceptance  into 
a  marketplace,  the  lead  time  for  a  developmental  concept  to  reach  the  warfighter  can  be 
greatly  reduced. 


Figure  5  provides  an  overview  of  the  widget  governance  tool  that  governs  a  widget, 
beginning  with  its  initial  submission  to  the  widget  governance  process  to  its  acceptance  into 
the  operational  environment  where  it  is  becomes  available  for  use  by  the  warfighter. 
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Figure  5.  Widget  Governance  Process 
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Developers  provide  widgets  to  PORs,  which  expose  capabilities  in  a  widget 
framework  (1).  The  widgets  must  meet  entrance  criteria  for  introduction  to  the  test  and 
integration  (T&l)  environment  (2),  which  includes  the  source  code,  descriptive  metadata, 
configuration  documentation,  and  developer  testing  results  for  the  target  production 
environment.  Applying  COMOPTEV/ODAA-approved  processes,  the  widget  passes  through 
a  number  of  manual  and  automated  tests  to  ensure  suitability  for  the  production  marketplace 
environment  (3).  Upon  review  of  the  test  results,  which  verify  that  the  widget  meets  the  exit 
criteria  (4),  the  widget  is  approved  to  be  introduced  into  the  marketplace  operational 
environment  (5)  and  is  made  readily  available  to  the  warfighter. 

Figure  6  is  a  detailed  process  flow  for  the  widget  governance  tool. 
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Figure  6.  Detailed  Widget  Governance  Process  Flow 

A  widget  submission  package  (WSP)  is  submitted  (1),  which  contains  source  code 
and  documentation  of  the  widget  and  application  programming  interface  (API),  as  well  as 
metadata  describing  the  function,  user  guidance,  characteristics,  boundaries  and 
deployment  locations,  preferred  browser  and  system  configuration,  installation  instructions, 
and  dependencies.  Developer  functional,  IA,  and  integration  test  reports  are  also  included, 
as  well  as  a  mobile  code  risk-mitigation  strategy  and  a  statement  that  the  widget  has  been 
developed  in  accordance  with  Mobile  Code  Developer’s  Guidance  and  a  Security  Technical 
Implementation  Guide  (STIG)  report.  All  required  components  of  the  WSP  are  indexed  for 
ready  reference.  If  the  package  does  not  pass  the  acceptance  test  (1 ),  a  report  of 
deficiencies  is  provided  and  the  submitter  is  provided  the  opportunity  to  edit  and  correct  the 
submission  (2).  If  the  WSP  passes  the  acceptance  sub-process,  the  package  is  provided  for 
functional,  IA,  and  integration  testing  sub-processes  in  the  T&l  environment  (3). 

The  functional,  IA,  and  integration  testing  is  conducted  in  parallel  to  the  greatest 
extent  possible  in  order  to  optimize  testing  resources  and  make  the  procession  of  the  WSP 
through  the  process  efficient  (4).  Functional  testing  will  focus  on  the  proper  operation  of  the 
widget  in  generating  the  desired  output  in  a  widget  as  described  by  the  POR.  Integration 
testing  will  concentrate  on  how  well  the  widget  performs  in  the  marketplace  environment 
(e.g.,  with  the  widget  framework,  identity  management  solution,  etc.)  and  also  amidst  other 
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widgets.  IA  testing  will  ensure  that  the  widget  meets  OWF  standards,  that  backend  services 
and  data  inherit  configuration  attributes  from  their  accredited  parent  environments,  that 
information  is  exchanged  over  a  secure  channel,  and  that  the  widget  operates  in  a  manner 
that  ensures  an  acceptable  level  of  security.  Some  tests  will  be  conducted  manually  by  the 
T&l  test  team,  but  automation  is  desired  to  the  greatest  extent  possible  to  decrease  the 
amount  of  time  and  manual  effort  required  to  designate  a  widget  suitable  for  the  operational 
marketplace  environment. 

Upon  completion  of  the  preceding  tests,  the  results  will  be  aggregated  and  compiled 
for  the  approval  board  sub-process.  The  board  may  determine  that  a  WSP  needs  to  be 
returned  to  the  T&l  test  team  if  the  results  did  not  demonstrate  acceptable  functional,  IA,  or 
integration  testing  results  (5).  A  widget  may  also  be  ordered  to  be  reworked  by  the 
developers  if  major  deficiencies  exist  that  must  be  corrected  prior  to  deployment  to  the 
operational  marketplace  environment  (6).  Additionally,  a  WSP  may  be  rejected  if  the  content 
rendered  or  output  of  the  widget  is  deemed  to  be  inappropriate  or  of  no-added  value  in  the 
marketplace  environment  (7),  or  approved,  making  it  available  to  the  warfighter  in  the 
production  marketplace  environment  (8). 

Conclusions 

The  DoD  can  no  longer  continue  down  its  current  acquisition  path  providing 
yesterday’s  solutions  to  meet  today’s  immediate  needs.  The  DoD  must  modify  its  view  of 
acquisition.  As  technology  is  constantly  evolving  and  improving,  the  DoD’s  struggle  to  keep 
up  with  the  latest  capabilities  hinders  it  with  lengthy  acquisition  schedules  and  unsustainable 
costs.  The  current  commercial  trend  of  delivering  small,  lightweight,  mobile  applications  to 
an  application  store  has  allowed  industry  leaders  to  provide  a  consistent  stream  of  new 
capabilities  to  their  customers.  The  DoD,  however,  has  struggled  to  adopt  this  notion  of 
rapid  fielding  of  capabilities. 

The  future  of  warfare  is  information  dominance  and  speed  to  capability.  Lightweight 
web  applications  can  supply  the  warfighter  with  valuable  information  that  can  be  developed 
in  a  short  period  of  time  since  they  are  composed  of  a  generally  small  amount  of  code.  With 
shortened  development  times,  immediate  user  needs  can  be  addressed  and  satisfied  more 
quickly.  Widgets  provided  by  an  already-accredited  POR  do  not  need  to  undergo  the 
certification  and  accreditation  processes  that  lengthen  acquisition  schedules  and,  ultimately, 
consume  costs.  New  widget  technologies  and  smaller  testing  efforts  that  make  them 
available  within  an  application  store  will  introduce  a  paradigm  shift  in  the  development  and 
delivery  of  capabilities  to  the  warfighter.  The  PEO  C4I  Marketplace  and  accompanying 
Widget  Governance  Tool  will  provide  this  widget  technology  in  a  cost-effective  and 
expedient  fashion,  while  ensuring  trusted  and  secure  information  capabilities. 
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T  The  commercial  world  has  shifted  focus  to  small  flexible 
mobile  code  via  widgets  and  other  mobile  applications 

T  IBM’s  Global  Technology  Outlook  for  2006  noted  an 
impending  shift  toward  “situational  applications” 

■  Rapidly  evolving  software  development  paradigm 

■  A  driving  force  in  delivery  of  web-based  dynamic  content 

T  Using  standard  acquisition  processes  in  DOD,  by  the  time 
new  solutions  are  deployed  to  the  warfighter,  the 
technology  is  obsolete 

T  Several  programs  within  the  DOD  have  started  similar 
initiatives  that  hold  the  promise  of  reducing  the  “heavy 
lifting”  required  as  part  of  the  current  acquisition  process 
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▼/  Problems  with  the  Defense  Acquisition  System 

T  Major  defense  acquisition  programs  continue  to  experience 
significant  cost  growth  and  schedule  delays 

■  In  a  2008  report  to  Congress  regarding  an  evaluation  of  95 
MDAPs,  the  GAO  noted 

-  $296  Billion  in  cost  growth 

-  Average  schedule  delays  of  22  months 

T  Excessive  development  and  update  cycles 

■  Private  sector  cycles  are  12-18  months 

■  Defense  IT  systems  routinely  require  48  -  60  months 
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Industry  Approach 


T  Current  acquisition  rules  required  for  POR  systems  have 
caused  a  major  gap  between  the  technology  available  to 
the  warfighter  and  that  which  is  available  commercially 

■  Exacerbated  by  capabilities  of  smartphones  and  tablet  computers 

■  Greater  gap  for  younger  service  members  raised  on  commercial 
technologies  and  then  forced  to  use  outdated  systems 

T  Apple  and  Google  are  both  known  for  their  mobile 
operating  systems  and  the  applications  (or  “apps”)  that  run 
on  them 

■  Both  companies  have  “app”  stores  that  allow  developers  to  rapidly 
deploy  applications,  but  they  do  so  in  very  different  ways 
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T  DON/DOD  lacks  a  lightweight  web  application  test  and 
integration  (T&l)  environment 

■  Needed  to  model,  test,  exercise,  and  perform  certification  and  accreditation  of 
widget  capabilities 


▼  A  Widget  T&l  environment  is  required  for  widget 
technology  development  throughout  the  DOD 

T  A  DON/DOD  widget  T&l  environment  is  needed  that 
incorporates  the  unique  and  common  aspects  of  Navy 
widget  environments 
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What  is  a  Widget? 


T  Lightweight 

▼  Single-purpose 
T  User  configured 

T  Web-enabled  application 

T  Provides  summary  information  or  a  limited  view 
into  a  larger  application 

▼  Also  used  alongside  related  widgets  to  provide 
an  integrated  view 
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▼  A  platform  that  offers  infrastructure  services  to  simplify 
the  development  of  workflows  and  presentation-tier 
application  integration 


It  is  also  a  layout 
manager  for  the 
operation  of 
widgets  on  a 
single  web 
page 


User  s  Personalized 
Workspace 


Ozone  Widget  Framework 


Search  for  Widgets 
to  add  to  Personal 
Workspace 


Add  Widgets  to  Marketplace 


9 


Oom/n 


SPAWAR 


DOD  Widget  Efforts 


T  DOD  Chief  Information  Officer  Storefront  Pilot 
T  National  Security  Agency  Denver  Store 
▼  Global  Command  and  Control  Systems  -  Joint 

■  Joint  Command  and  Control  Common  User  Interface  (JC2CUI) 

■  Integrated,  Imagery  and  Intelligence 

T  Command  and  Control  Rapid  Prototyping  Continuum 

T  Integrated  Intelligence  Architecture 

T  Defense  Common  Ground  Station  Army  &  Army  Mission 
Command 

T  Defense  Intelligence  Information  Enterprise  Framework 
(Di2E-F)  Quick  Response  Capability 
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SPAWAR 


Widgets  in  Action 


▼  GCCS-J  I3  Sample  Widgets  ▼ 

■  I3  Common  Geospatial  Display 
Widget 

■  I3  Vessels  of  Interest  Widget 

■  I3  Maneuver  Unit  Widget 

■  I3  Latest  DMOB  Equipment 
Widget 

■  I3  Naval  Activity  Widget 

■  I3  Channel  List  Widget,  I3  Blue 
Forces  Widget 

■  l3AOB  Widget 

■  I3  Recent  Activity  Widget 

■  I3  Targeting  Widget 

■  13  Naval  Order  of  Battle 
Tracking 

■  13  Motion  Imagery  Viewer 


Sample  Widgets 


DCGS-A  Sample  Widgets 

■  Common  Admin  Widget 

■  Common  Help  Widget 

■  Common  Query  Widget 

■  Common  Map  Widget 

■  Coordinate  Conversion  Widget 

■  DIB  Query  Widget 

■  plus  over  50  additional  widgets 

-  Weather 

-  HUMINT 

-  IMINT 

-  Geospatial 

-  Alerting  tools 
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\  ▼  /  Widgets  in  Action  -  Engineering  at  the  Edge 

9* 


▼  Seeks  to  use  OWF  and  widgets  to  reduce  an  operational 
command’s  dependency  on  end-to-end  systems 
development  efforts  that  need  years  to  develop  and  may 
not  address  requirements  that  have  emerged  since 
development  started 

T  OWF  has  enabled  the  DOD  community  to  decouple  the 
User  Interface  and  visualization  components  from  the 
rest  of  the  architecture 
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Widgets  in  Action  - 
Widget  Developer’s  Toolkit 


Widget 


Toolkit 


C4  Widget  Cookbook 

-  Guidance  Recipes 

-  READMEs 

C4  Widget  SDK 

-  Scripts 

-  Core  Widgets 

-  Widget  Templates 

-  Code  Demos 

-  Configured  Environment 

-  Test  Data 

C4  Widget  Training 

-  Boot  Camp 

-  Online  Tutorials 
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Widgets  in  Action  - 
Widget  SDK  Test  Environment 
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SPAWAR 


A  distributable  development,  test,  and  debug  environment 

for  widget  development. 


Code  Demos 

-  Example  code  and  tutorials  for 
developing  various  Web  and  OWF 
widget  concepts. 

Configured  Environment 

-  A  package  with  the  C4  Widget  SDK 
components  and  an  environment  with 
OWF,  OMP,  various  testing  and 
debugging  tools,  and  OWF  extensions 
already  installed  and  configured. 

Test  Data 

-  Unclassified  data  for  widgets  and 
services  to  utilize  in  order  to  better 
prototype  and  test  functionality. 
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\W€'J  Widgets  in  Action  -  Widget  Training 

9* 


Provide  a  means  for  developers  to  gain  remote  and  local 
experience  with  widget  development  and  using  the  toolkit. 

Boot  Camp 

-  2  day  instructor-led  presentations  and  hands  on 
examples  with  widget  development  and  toolkit 
concepts  and  products  geared  for  developers  of 
all  skill  levels. 

Online  Tutorials 

-  Observe  &  Take  Action  (OTAs)  videos  using  the 
toolkit  and  developing  widgets. 

-  Recorded  boot  camp  tutorials  via  WebEX/DCO 
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r irements 


Storefront 

Application 

Marketplace 


Adopt 

Capabilities 


Response 


DoD  Service  Providers 
Environment 

3**  *A 

i  nces  M 


Request 


Intelink 


Defense  Travel  System 

A  New  Era  of  Government  Travel 


DCOik 


DoD  User  Environment 
■Enterprise  Application  Framework 
■Productivity  applications  and  widgets 
■Collaboration 
■Social  Networking 
■Publish/Discover 
■Search/Retrieve 


Feedback  and 
New  Opportunities 


DoD  User 
Environment 


Deploy 

Capabilities 


Developer  Environment 

Project  hosting 
Collaboration/SNS  tools 
APIs/SDKs  cftpn(=At 

Documentation  ^ 

CM  tools 
C&A  support 
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PEO  C4I  Storefront  &  Navy  Cloud 


User  Access 


PEO  C4I  Storefront 

Missions  Support  Modules:  Widgets, 
Application,  Services 


r^\  /~zR 


Utility  Cloud 
Data  Cloud 

Storage  Cloud 

CANES 

Accelerating  Acquisition  To  Enable 
Rapid  Fielding  of  New  Capabilities 


PEO  C4I  Storefront 


Missions  Support  Modules: 
Widgets,  Application,  Services 


Widget  &Apps 
T&l  Environment 


Agile  Widget/App 
Approval 


Utility  Cloud 

Test  widget 


Enterprise 

Cloud 


Imagery 

Storage  Cloud 


Afloat 
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.▼/  Widget  Lifecycle  and  Operational  Architecture 


GCCS  I3  Server 
ATO 


L/ser 


OM/F  Server 
ATO 


OPS  Architecture 

Storefront 
Ozone  Marketplace 
ATO 


18 


•iV®  lnfo 


0om/n 


SrWAR 


3 

9  <» 

o 

o 

$ 


Proposed  Widget  Governance  Process 


Development 


i 


Developers 


Development 

Repository 

SDK 

API 

Source  Code 


POR 

Sponsored 

Widgets 

A 

V 

Entrance 
Criteria 
Prerequisites 
Source  Code 
Documentation 


Test  &  Integration 
Processes 


Exit 

Criteria 

Met: 

Widget 

Approved 


OPTEV  /  ODAA  Approved  Processes 


Integration 

Testing 


Acceptance 


IA 


Functional 

Testing 


Approval 

Board 


Operational  Repository 
Widget/Service 
Warehouse 


Storefront 

Approved 

Widgets 


Trusted  Environment 


Warfighter 

Deployment 
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Metrics 

Collection 


Exposure 


Discovery 


Configuration 

Management 


Metrics 

Collection 
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PE0-C4I  Storefront/Marketplace 


PEO-C4I 

Marketplace 


* 

Market 


Types:  Widget- 


PEO  C4I  Marketplace 


Marketplace  is  a  registry  of  applications  and  services  similarto  a  commercial  application  store  where  Users  can  submit  and  interact 
with  software  components  or"Listings"  such  as  widgets,  plugins,  REST  &.  SOAP  services,  web  apps  and  more. 


Type 

Any  Type 


Showing  1-3  of  3  results 


iiC° 


Category 


Rating 


Listing  Manager 

My  Listings 


Owner:  Aaron  Bajek 

Modified:  10/1 7/2011 


Sort  by  |  most  relevant 


American  Forces  Press  Service... 

Type:  Widget 

Version:  1.0  Release:  1 0/1 7/201 1 


The  AFPS  News  widget  makes  it  easy  for  anyone  with  a  Web  site,  blog  or 
social  networking  page  to  carry  the  latest  AFPS  articles  on  their  site.  The 
widget  allows  Web  sites  to  carry  a  steady  stream  of  the  latest  official 
news  and  information  about  the  Defense  Department. 

Categories:  Category  A 


Items  per  page  5  v 


(0) 


[ACTIVE] 


Department  of  Defense  Photogr... 

Type:  Widget 

Version:  1.0  Release:  1 0/1 7/201 1 

The  Department  of  Defense  Photography  widget  brings  the  latest  and 


(0) 
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PE0-C4I  Storefront  Widget  Samples 


PEO-C4I  Storefront 


Dynamic  Page  -  Highest  Possible  Classification  is  UN  CLASSIFIED/ /FDR  OFFICIAL  USE  ONLY 


|  DESKTO  P_4e5a  f464-a  a51  -1 B33-. . .  |  MDAGMMS 


Welcome  Administrator  |  Logout  I  Hide 
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Launch  Menu  Share  Customize 


Eventing  .H— 
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T  DOD  must  modify  its  acquisition  philosophy  to  get 
new  capabilities  in  the  hands  of  the  warfighter 

■  Light  weight  mobile  applications 

■  Access  to  services  and  data  sources 

■  Streamlined  processes  for  accredited  PORs 

-  Allows  rapid  fielding  of  associated  Widgets 

■  PEO  C4I  Marketplace  and  accompanying  Widget  Governance  Tool 

-  Cost  effective  and  expedient 

-  Provides  trusted  and  secure  capabilities 


T  The  future  of  warfare  is  information  dominance  and 
speed  to  capability  can  provide  the  tactical  or  strategic 
advantage  our  warfighters  need 
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Problems  with  the  Defense  Acquisition  System 


Analysis  of  DOD  Major  Defense  Acquisition  Program  Portfolios  (fiscal  year  [FY]  2008  dollars) 

FY  2000 

FY  2005 

FY  2007 

Portfolio 

Portfolio 

Portfolio 

Portfolio  size 

Number  of  programs 

75 

91 

95 

Total  planned  commitments 

$790  Billion 

$1.5  Trillion 

$1.6  Trillion 

Commitments  outstanding 

$380  Billion 

$887  Billion 

$858  Billion 

Portfolio  performance 

Change  to  total  RDT&E  costs  from  first  estimate 

27  percent 

33  percent 

40  percent 

Change  in  total  acquisition  cost  from  first  estimate 

6  percent 

18  percent 

26  percent 

Estimated  total  acquisition  cost  growth 

$42  Billion 

$202  Billion 

$295  Billion 

Share  of  programs  with  25  percent  or  more  increase  in 
program  acquisition  unit  cost 

37  percent 

44  percent 

44  percent 

Average  schedule  delay  in  delivering  initial  capabilities 

16  months 

17  months 

21  months 

Source:  GAO  analysis  of  DOD  data. 
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Industry  Approach 


T  The  Apple  App  Store  contains  Apple-created  applications 
and  third-party  mobile  applications  are  available  where 
they  may  be  discovered  and  downloaded  by  the  customer 

■  Apple’s  app  approval  process 

-  Ensures  submitted  applications  are  reliable  &  perform  properly 

-  Adhere  to  Apple’s  requirements  regarding  appropriate  content 

■  Developers  are  provided  App  Store  Review  Guidelines 

■  Upon  submission  for  review 

-  Applications  are  checked  for  compliance 

-  Apps  must  comply  with  technical,  content,  and  design  criteria 
before  they  are  made  available  in  the  App  Store 

-  Appeals  of  rejected  applications  are  sent  to  an  App  Review 
Board 
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Industry  Approach 


T  Developers  of  apps  for  Google’s  Android  OS  have  two 
deployment  options 

■  The  Android  App  Section  of  Google  Play  (formerly  Android 
Marketplace)  and  the  Amazon  Appstore 

■  Google  Play  does  not  monitor  uploaded  applications 

-  It  does  place  restrictions  on  application  content,  network  usage, 
spam,  and  processing  payments 

-  If  an  application  violates  restrictions,  Google  can  remove  the 
application  from  Google  Play 

■  Apps  submitted  to  the  Amazon  Appstore  are  tested 

-  Ensure  that  apps  work  as  described  in  the  product  description 

-  Do  not  impair  the  functionality  of  the  user’s  mobile  device 

-  Comply  with  Appstore  Distribution  Agreement  and  content  guidelines 
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T  A  DON/DOD  widget  T&l  environment  is  needed  that 
incorporates  the  unique  and  common  aspects  of  Navy 
widget  environments 

■  Will  replicate  the  environment  that  connects  directly  to  the  Global 
Information  Grid  (GIG) 

■  Also  replicate  Navy,  afloat,  ashore,  deployed  and  limited 
communication  environments 

-  Hardware,  software,  and  associated  configurations 

■  Creates  a  Navy  trusted  source  environment  for  widget  transition 
to  the  fleet 

■  Help  to  develop  more  effective  and  efficient  techniques, 
processes,  and  procedures  to  achieve  speed  to  capability 
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T  Providing  standards,  specifications,  processes,  and  a  T&l 
environment  will  ensure  proper  products  are  discoverable 
by  the  warfighter 

■  Enable  software  based  mission  capabilities  to  be  released  in 
weeks  rather  than  months  or  years 

T  A  DOD  marketplace  or  application  store  is  also  needed  to 
rapidly  and  securely  field  software  capabilities  to  the 
warfighter  in  the  form  of  widgets  and  web  applications 

T  Widgets  provide  a  technological  capability  to  foster  this 
rapid  fielding  ability 

■  Widgets  provide  very  specific  functionality 

■  Generally  require  a  relatively  small  amount  of  software  code 

■  Created  on  a  relatively  short  timeline 
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Widgets  in  Action  - 
Widget  Cookbook 


Provide  toolkit  product  documentation  and  recipes  to  help 
quickly  design  and  implement  widgets  for  the  OWF. 


Guidance  Recipes 

-  Preface 

-  Ch.  1:  The  Widget  Environment 

-  Ch.  2:  User  Experience 

-  Ch.  3:  Widget  Resources  &  Services 

-  Ch.  4:  Inter-Widget  Communication 

-  Ch.  5:  Core  Widgets  &  OWF 

Extensions 

-  Ch.  6:  Security 

-  Ch.  7:  Testing  &  Debugging 

-  Ch.  8:  Performance 

-  Ch.  9:  Packaging  &  Deployment 

-  Ch.  10:  Marketplace 

-  Appendices 


READMEs 

-  Documentation  for  relevant 
Toolkit  products  (e.g.  core 
widgets). 
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Widgets  in  Action  - 
Widget  SDK  Tools 


Provide  tools  to  help  simplify,  automate,  and  reduce 
duplication  of  effort  in  the  build,  package,  and  deploy  process. 


Scripts 

-  Tried  and  tested  scripts  to  prepare  and  deploy 
widgets  for  use  in  the  OWF. 

Core  Widgets 

-  Provide  ready-to-go  widgets  that  have  common 
functionality  and  are  used  almost  all  of  the  time 
(e.g.  map  widget)  along  with  instructions  for  use. 

Widget  Templates 

-  Provide  a  number  of  configurable  widget 
templates  for  developers  to  fill-in-the-blanks  and 
make  their  own. 
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\  ▼’/  Widget  Governance  Tool 


rework  [  deficiency_description  =...] 


retest  [  integration=true;  ia=false;  functional=false] 
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